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Aim 

To use the Lagrange multiplier method to derive the 
distribution of microstates for distinguishable particles. 

Objectives 

1. To explain what distinguishable particles are. 

2. To count microstates for particles with more than two 
energy levels. 

3. To describe and use the Lagrange multiplier method. 

4. To explain Boltzmann's postulate for entropy. 

5. To derive the Boltzmann distribution. 
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2.1 Microstates and Macrostates 
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Distinguishable particles 



Two particles are distinguishable if you can tell them apart. A 
1 p coin and a 2 p coin would be distinguishable particles. 

If you have two identical coins, they may be indistinguishable. 
However, you may stick a label on one, then they become 
distinguishable particles again. 

You cannot stick labels on identical atoms or electrons. So 
those are indistinguishable particles. 

However, there is a special case for atoms. When atoms are 
fixed in a solid, you can use their difference in location to tell 
one atom from another. 

Just remember the position, and you can always get back to the 
same atom. So atoms in a solid are distinguishable particles. 
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Particles with more than two energy levels 



We have previously looked at the tossing of coin, and used this 
as an analogy for a particle with two energy levels. 

A real particle could have more than two energy levels. We 
need to look at the problem of how to specify the macrostates, 
and how to count them. In order to do so, we need to develop 
some notations. We shall start with the coin. 

A coin has 2 sides: head and tail. Suppose we toss the coin 
N = 2 times. The result is ni = 2 heads, and = 0 tails. Now 
lets rephrase that. 

Consider a particle with 2 energy levels: e\ and e^. Suppose we 
have N = 2 particles. And there are n\ = 2 particles at energy 
level si, and = 0 at e^. 
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Particles with more than two energy levels 



So for the particle with energy levels e\ and s^, we have 
specified the macrostate using numbers (711,712) = (2, 0). 

So we can specify the macrostate using the number of 
particles at each energy level. 
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Particles with more than two energy levels 



Lets write down all the macrostates and microstates for the 
coin: 

(ni, n 2 ) Microstates 

(2,0) [H,H] 

(0, 2) [T, T] 

(1, 1) [tf,T], [T,H] 

and rephrase this for the particle: 

(ni, 77-2) Microstates 
(2, 0) 

(0, 2) [e 2 ,e 2 ] 

(1, 1) [si,s 2 ], [s 2 ,^i] 

Note, for example, [si,s 2 ] would mean that particle 1 is at level 
s\ and particle 2 is at level s 2 . 
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Particles with more than two energy levels 



We are ready to take the next step. 

Consider a particle with 3 energy levels: e\, £2 ancl £3- Suppose 
there are N = 2 particles. And there are n\ = 2 at energy level 
e \y n 2 — 0 at £2, and 713 = 0 at S3. 

We have again specified the macrostate using the number of 
particles at each energy level (ni, 712,713) = (2, 0, 0). 
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Particles with more than two energy levels 



Lets write down a useful relation: 

N = n\ + ri2 + ns 

The total number of particles is equal to the sum of the 
particles at each level. 
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Particles with more than two energy levels 



These are all possible macrostates and microstates: 

(ni, ri2, n 3) Microstates 

(2, 0, 0) [si, si] 
(0, 2, 0) [s 2 ,s 2 ] 
(0, 0, 2) [s 3 ,s 3 ] 

(1, 1, 0) [£i,S 2 ], [£2,£l] 
(0, 1, 1) [£ 2 ,£3]> [ £ 3^2] 

(1, 0, 1) [ei,e 3 ], [s 3 ,si] 

For example, for the macrostate (1, 0, 1), we can have 2 
microstates: 

[si,s 3 ] means particle 1 at si and particle 2 at s 3 . 
[s 3 ,si] means particle 1 at s 3 and particle 2 at e\. 
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Particles with more than two energy levels 



Another way of representing these microstates is to use a 
picture like this: 

Macrostate Possible Microstates 

[e 1iE3 ] fee,] 




Each of the red circles represents a particle. 
The full list is shown on the next silde. 
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Particles with more than two energy levels 
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Particles with more than two energy levels 



It quickly gets complicated if we increase the number of levels. 
Fortunately, we usually need to know the number of 
microstates, but not the actual arrangements. 

There is a formula for this number. We return to the coin 
tossing example: 

"A coin has 2 sides: head and tail. Suppose we toss the coin 
N = 2 times. The result is n\ = 2 heads, and = 0 tails." 

We have already seen the formula previously. The number of 
microstates for the macrostate (ni,ri2) is 



It is simply the number of ways of putting TV different objects 
into 2 boxes. 
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Particles with more than two energy levels 



So we can think of the particles as different objects, and the 
energy levels as boxes. With this idea, the formula can be 
easily extended to the particle example: 

"Consider a particle with 3 energy levels: e\, £2 ancl £3- 
Suppose there are N = 2 particles. And there are n\ = 2 at 
energy level e\, = 0 at £2, and 713 = 0 at S3." 

We want to know the number of microstates in the macrostate 
(ni, 712,713). This would be the number of ways of putting N 
different objects into 3 boxes, since there are 3 energy levels. 
The answer is 

n= m 
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Particles with more than two energy levels 



The formula generalises to of energy levels, for any number 
particles: 

TV" 

Q = 



ni\n2 l -n^\...rii\... 
or 



Ui 



We now have to consider how we want to use this formula. 



Previously, we have drawn an analogy between coin tossing and 
atoms in a cup of tea. There is a parallel between the number 
of heads, and the total energy of the atoms. 



PHYS393/Hock 



15 



2009-2010 



Particles with more than two energy levels 



We have seen that when there is a very large number of tossing 
or of atoms, the most probably macrostate becomes almost the 
only macrostate. 

If we can find this macrostate, we would know the number of 
particles at each energy level. This would allow us to find the 
total energy, the heat capacity, and other thermodynamic 
quantities, which is what we set out to do. 

So we want to use the formula for the number of microstates, 

ft = , 

Ui rii\ 

to find the most probable macrostate. 
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Particles with more than two energy levels 



In order to do so, there are 2 more equations we need to 
include. 

The total number of particles is N. The macrostate is specified 
by the number of particles at each energy level: 
(ni, r&2, rii, ...)■ So the total number must the the sum of the 
particles at each level: 

i 

We consider an isolated system, e.g. a cup of tea that is very 
well insulated from the surrounding. Then the total energy U 
of the particles would remain constant. 

There are n\ particles at energy level e\, so the total energy of 
these particles is n\s\. Adding up the energy of the particles at 
all energy levels, we have the energy of the whole system: 

i 
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Particles with more than two energy levels 

Lets write down all the equations we have. 

For a macrostate n iy specified by the number of particles in 
each energy level: (ni,n2,...): 

The number of microstates is given by 



Ui rii 1 



We assume that the total number of particles is fixed 



Hi 



i 

Assuming that the system is isolated, the total energy is also 
fixed: 



U = J2 n i € i 
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To find the most probable macrostate 



Q is a function of the variables (ni,n2, ...)■ 

There are 2 conditions, or constraints, on these variables: 

(i) the total number of particles is a constant, and 

(ii) the total energy is also a constant. 

In order to maximise Q, we can use the method of Lagrange 
multipliers. 
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To find the most probable macrostate 
The procedure is as follows: 

1. We maximise InQ instead of Q. The result would be the 
same because when InQ is a maximum, so is Q. 

2. The reason is that InQ can be simplified for very large TV 
using the Stirling's theorem. 

3. Write down the Lagrange function In Q + AiTV + X2U. 

4. Ai and A2 are new parameters called Lagrange multipliers. 

5. Differentiate this with respect to every n^. 

6. Set the derivative to zero and solve for n^, Ai and A2. 

7. This gives the macrostate (711,712, ...)■ 
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2.2 Lagrange Multipliers 
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A brief explanation on Lagrange Multipliers 



My) 




X 



1. Suppose you want to find the highest point along a road 
that does not quite go over the top of a hill. 

2. The highest point on the road must be just touching one of 
the the contours of the hill. 
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Look at the contour map 




1. Represent the road as a graph g(x,y) = 0 

2. This turns the road itself into a contour of the function 
g(x,y). It is the contour when this function is zero. 

3. At the highest point the contours touch, the normals to the 
contours are aligned. 
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Gradient vector of the hill. 



1. Suppose the hill is represented by the function f(x,y). 

2. It is possible to find a vector with the following properties: 

• pointing in the direction of greatest slope, 

• with a magnitude equal to the gradient of that slope. 
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>x 

1. This means that the vector must be perpendicular to the 
contour. 

2. This vector, also simply called "gradient", is calculated 
using Vf(x,y) = (U,U) 
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>x 

1. At the point where the contours touch, the gradient vectors 
of both f(x,y) and g(x,y) are aligned. 

2. This means that we can find A such that 

Vf(x,y) + \Vg(x,y) = 0 (1) 
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The Method of Lagrange Multiplier 



1. Problem: Maximise f(x,y) subject to the constraint 
g(x,y) = 0. 

2. Solution: Write down the Lagrange function 

f(x,y) + \g(x,y) and find its gradient. The parameter A is 
called the Lagrange multiplier. 

3. We have 3 equations now. The gradient gives 2 equations: 



df(x,y) 



+ A 



dg(x,y) 



0 



(2) 



dx 



dx 



df(x,y) 
dy 



+ A 



dg(x,y) 
dy 



0 



(3) 



and the constraint gives one: 



(4) 



4. Solve for x, y 



and A. 
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A Leap to N Dimensions 



1. The method can be extended to any number of variables 
and constraints. Suppose we have a function of n variables, 

f\X\, X2, X n ) . 

2. Suppose we have two constraints, g(xi, x^, x n ) = 0 and 

h(x\, X2, Xn) = 0. 

3. Write down the Lagrange function 

f(x\, X2, x n ) + Ai^(xi, X2, x n ) + \2h(x\,x2, and 
find its gradient. Note that each constraint has one 
Lagrange multiplier. 

4. The gradient gives n equations, and the constraints give 2. 
We have n unknowns from the variables xi,x2, ...,x n and 2 
from the multipliers Ai and A2. 

5. Solve the equations for all the unknowns. 
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If we need to find the maximum of a function, like Q, subject 
to constraints like fixed TV and fixed U, we use the Lagrange 
multipliers. 

First, we use InQ instead of Q. When one is maximum, so is 
the other, so it makes no difference for this problem. The 
Stirling's theorem is used to simplify InQ. 

Then we write down the Lagrange function In Q + AiTV + X2U 
and maximise with respect to n^. 
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Stirling's Theorem 



The Stirling's Theorem is used for large factorials. 

For real objects, TV may be the number of atoms. This could 
be as big as 10 30 , and it takes a long time to calculate. 

Fortunately, there is a simple, approximate formula that is very 
accurate for large numbers: 

In TV! ^ TV In TV — TV 
This formula is the Stirling's Theorem. 
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The algebra 



We need to maximise 

\nn + X 1 N + X 2 U 
In order to do so, we need to differentiate this. Start with 

N = n\ + n2 + ... + ni + ... 

Differentiating with respect to ni gives 

dN _ 



'i 

Next, we differentiate 

JJ = mei + n 2 C2 + ••• + riiei + 

This gives 



dU 
drii 



Next we need to differentiate InQ 
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The algebra 



Start with the formula 



Then take the logarithm, 

In Q = In TV! — In ni! — In — ... — In — ... 
and apply Stirling's theorem: 

In Q = TV In TV— TV— (ni In n\— n\) — {n^ In n^— n^) — — (m In n^— n^) — ... 
Differentiating, we get 

= in TV — In rii 

dni 

Note that in this differentiation, TV is to be treated as a 
function of and not a constant. The condition that TV is a 
constant should be imposed afterwards. 
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The algebra 



These are what we have obtained 

dN 

= 1 



drii 
dU 

drii 
<9lnQ 

drii 



= £i 



= In TV — Inn, 



To maximise the Lagrange function In Q + XiN + \2U, we need 

to differentiate this and set the derivative to zero: 

<9lnQ dN dU 
h Ax h A 2 = 0 

oni oni oni 

Substituting the above results, we get: 

In TV - In rii + Ai + X 2 Si = 0 



PHYS393/Hock 



33 



2009-2010 



The most probable macrostate 



We have obtained 

In TV - In rii + X 1 + X 2 Si = 0 
Soving for gives the solution: 

rii = Aexp(A 2 q) 

where A = TVexp(Ai). 

Next, we need to determine Ai, or A, and A 2 . 
A can be obtained using 

the condition that the total number of particles is N. This is 
called the normalisation condition. Substituting, we get 

N = A^exp(A 2 e < ). 

Rearranging this gives 

A= N 

Eexp(A 2 €i) 
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Boltzmann's postulate 



Next, we need to find A2. In principle, we can use the next 
constraint 

Substituting the solution 

rii = Aexp(A 2 q) 

we get 

U = J2 Ae i exp(A 2 q) 

Then solve for X 2 . 

Unfortunately, this equation is very difficult to solve. Instead, 
we need to take another approach. That is to make use of our 
knowledge of thermodynamics. 
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2.3 Boltzmann Postulate 
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Boltzmann's postulate 



The purpose of doing all these calculations is to find a way to 
calculate thermodynamic quantities like entropy and heat from 
first principles. 



Instead of solving for A2 directly, we can try and find it in terms 
of known quantities in thermodynamics. We shall start by 
stating the result: 

A 2 = - 1 



k B T 

where k B is the Boltzmann constant and T is the temperature, 



There are many ways to determine this. We shall look at the 
following approach: 



1. Use entropy to show that A2 is related to T. 

2. Use the ideal gas law to obtain the above equation. 
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Boltzmann's postulate 



To relate A2 to the entropy formula 

dQ = TdS, 

we look at the Lagrange function In Q + \\N + X 2 U again. 

Previously, we have taken the derivative to find the set of 
numbers (711,712,...) that would maximise the function. We 
have found the solution for ri^, except for the unknown A2. 

For small variations about this solution (711,712,...), the change 
in the Lagrange function would be zero, since it is at a 
maximum. So 

d(ln Q) + XidN + X 2 dU = 0 

Suppose we impose the condition that the total TV is fixed, but 
we give some heat dQ to the system so that U increases a little. 
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Boltzmann's postulate 



Since U increases, we are going beyond the original problem 
statement, in which U is fixed. We are going to modify the 
constraints on TV and U. 

Injecting the heat would cause the numbers (ni,ri2> •••) to 
change. Particles at the lower energy levels would be excited to 
higher energies. 

However, we are making use of the results from the original 
problem. We a small change from the point (ni,n2,...) where 
the Lagrange function is at a maximum. 

This means that that any change in the function should remain 
zero to first order: 

d(ln Q) + XidN + X 2 dU = 0. 
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Boltzmann's postulate 



By injecting the heat dQ, we have modified the constraint U. 

We shall require that the constraint on TV remain the same, 
that is the number of particles remain fixed. 

The numbers in the individual levels can change. For 
example, n\ may decrease slightly, and n 2 may increase, so that 
the energy U increases. 

Then dN = 0 and dU = dQ, and we have 

d(ln Q) + XidN + X 2 dU = d(ln Q) + X 2 dQ = 0 
Rearrange this to compare with the entropy formula 

dQ = d(lnQ) 

A 2 

dQ = TdS. 
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Boltzmann's postulate 



Comparing the 2 equations: 



gives 



This implies that 



dQ = d(lnfi) 

A 2 

dQ = TdS. 



TdS = -— d(lnfi) 
A 2 



1 

T — — 



k\ 2 
S = fclnQ, 

for some constant k. We can verify these by substitution: 

TdS = --^d(#ln Q) = -— d(ln Q) 

icA 2 A 2 

The constant /c simply make the expressions for T and S more 
general. 
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Boltzmann's postulate 



Therefore, we have shown that A 2 is indeed related to T. The 
value of the constant k would have to wait. 



We shall see later, in the lectures on ideal gas, that k is the 
Boltzmann constant 

k B = = 1.3807 x 10~ 23 J K -1 , 

N A 

where R is the ideal gas constant and N A is Avogadro's 
constant. So 

1 



A 2 = - 

k B T 

We now have the complete solution for the macrostate 
(ni,n 2 , ...) : 

ni = A exp(A 2 £j = Aexp ( 

k B T , 
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Boltzmann's postulate 



The most probable macrostate is given by 

m = A exp f — —— 

The following symbol is often used in statistical mechanics: 

1 



k B T 

so we would also see the macrostate in this form 

rii = Aexv(-(3si) 

where 

A= N 



Eexp(-/3ei) 

Note that we have not arrived at this by solving the problem 
mathematically. 
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Boltzmann's postulate 



We have assumed that our macrostate idea is an accurate 
description of a real system, and then we have used our 
knowledge of thermodynamics to find the unknown Lagrange 
multiplier A2. 

As we shall see in these lectures, the predictions of this formula 
has been verified by many experiments. It is from these 
empirical results that we can finally accept that the ideas about 
macrostates and microstates are a good description of how 
energies and distributed among atoms and electrons. 

The solution for the macrostate tells us how the number of 
particles are distributed in different energy levels: 




It is called the Boltzmann distribution. 
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Boltzmann's postulate 



We have arrived at the expression for the Lagrange multiplier 
A2 by comparing these 2 equations: 

Recall that when we compare these 2 equations 

dQ = d(lnQ) 

A 2 

dQ = TdS. 
we have arrived at 2 results: 

1 

T — — 



k\ 2 
S = fclnfi, 

One is an expression for the Lagrange multiplier A 2 . The other 
is something even more exciting: 

S = kg In Q. 

We have found a formula for entropy! 
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Boltzmann's postulate 



Historically, the formula 

S = kg In Q. 

was first postulated by Boltzmann. It is so important that it is 
engraved on his tombstone ... 
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Entropy 



As with the Boltzmann distribution, we know that the entropy 
formula 

S = k B \nn 

is correct because it works. Again, as we shall see in the 
following lectures, it agrees with many experiments when we 
use it to predict thermodynamic quantities. 

Even so, it helps to see that this formula, based on the number 
of microstates Q, agrees with some of the properties that we 
know about the entropy S: 

1. We know that entropy tends to increase. We know that a 
system tends towards the macrostate with the greatest number 
of microstates. 

2. We get the total entropy of two systems by adding. We get 
the number of microstates Q by multiplying, but the logarithm 
of multiplcation becomes addition. 
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A note on the recommended text 



If you are using the recommended text: 

Statistical Mechanics - A Survival Guide, 
by A. M. Glazer and J. S. Wark 

please note that the Lagrange multiplier method given on 
pages 15 and 16 is incorrect. 

Please use the version given in these lectures. 
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What we have learnt so far 



1. We study an isolated system of TV particles. We hope that 
this can be used to describe a real system of electrons or 
atoms. 

2. We assume that as long as the total energy U is fixed, the 
system is equally likely to be in any microstate. 

3. To find the most probable macrostate, we maximise InQ, 
where Q is the number of microstates in a macrostate specified 

by (ni,n 2 ,...)- 

4. We formulate this as a problem to find the answer 
(ni,n2,...) for which the function In Q is a maximum, subject to 
the constraints that TV and U are constants. 
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What we have learnt so far 



5. With the help of the formula dQ = TdS, we find that the 
most probable macrostate is the Boltzmann distribution, 

rii = Aexv(-Si/k B T). 

6. This also leads to a formula or postulate for entropy 

S = k B T\nn, 
which has to be verified by experiments. 

7. We assume that because TV is very large, the probability of a 
different macrostate from the Boltzmann distribution is 
extremely small. It should be possible to demonstrate this in 
the same way that we have done previously for the tossing of 
coins. 
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2.4 Exercises 
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Some exercises 



Exercise 1 

Minimise the function f(x, y) = x 2 + y 2 , subject to the 
constraint x + y — 2 = 0. 

(i) Do it mentally, or by inspection of the graphs. 

(ii) Do it using Lagrange multiplier. 
Do the results agree? 
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Mentally 




1. The contours of f(x, y) are just circles, e.g. x 2 + y 2 = l, 

x 2 + y 2 = 2, ... The circles get larger as the value increases. 

2. The constraint x + y — 2 = 0 is a straight line with 
intercepts x = 2 or y = 2. 
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Answer: Mentally ... 




1. By symmetry, f(x,y) is highest at the midpoint between the 
intercepts, where x = y = 1. 

2. Therefore, f(x,y) = l 2 + l 2 = 2. 
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Using Lagrange multiplier 



1. Let g{x, y) = x + y — 2. The constraint is then g(x, y) = 0. 

2. The Lagrange function is 

fix, y) + \g(x, y)=x 2 + y 2 + \(x + y- 2) (5) 

3. The gradient gives: 

1. Differentiate wrt 00 m 00 

2. Differentiate wrt y: 2y + X = 0. 

4. Solving with the constraint equation gives A = — 2 and 
x — y — 1, as before. 
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Some exercises 



Exercise 2 

Use your calculator to work out In 10!. Compare your answer 
with the simple version of Stirling's theorem (TV In TV — TV). How 
big must TV be for the simple version of Stirling's theorem to be 
correct to within (ii) 5% (ii) 1% ? 

Answer 

ln(10!) = 15.11 whereas 10ln(10)-10 = 13.03. There is less 
than 5% difference for N = 24 and less than 1% for B = 91. 
Stirling's approximation is quite accurate even for relatively 
small N. 
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Exercise 3 



Exercise 3 

Consider a 10000 distinguishable particles at room temperature, 
298 K. Suppose that each particle has 2 energy levels, 0.01 eV 
and 0.02 eV. Find the number of the particles in each energy 
level. ( Boltzmann's constant is 1.3807 x 10 -23 J K _1 .) 
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Exercise 3 



Answer 

Since the number is quite large, we assume that the probability 
at each energy level is given by Boltzmann's distribution 

Aexv(-s/k B T). 

At 0.01 eV, Aexp(-s/k B T) = 0.6778A 
At 0.21 eV, Aexp(-s/k B T) = 0.4594A 
The total is 0.6778A + 0.4594A = 10000. So A = 8794. 

Therefore the numbers are: 

at O.OleV, 0.6778 x 8794 = 5961; 
at 0.02eV, 0.4594 x 8794 = 4040. 
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Exercise 4 

Calculate the quantity k B T at room temperature, 298 K. Give 
the answer in eV. ( Boltzmann's constant is 1.3807 x 10 -23 J 
K _1 . Electron charge is 1.6 x 10 -19 C.) 

Answer 

In Joules, k B T = 1.3807 x 10~ 23 x 298 = 4.114 x 10~ 21 J. 

In eV, k B T/e = 4.114 x 10~ 21 /1.6 x 10~ 19 = 0.026 eV. 

Note: Since 1/40 = 0.025 is quite close to this answer, k B T at 
room temperature is often quoted as 1/40 eV. 
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Exercise 5 

Consider a large number of distinguishable particles, at 
temperature T.. Each particle has 4 energy levels 0, k B T, 2k B T 
and 3k B T. Calculate the fraction of particles in each energy 
level. 

Answer 

Since the number is quite large, we assume that the probability 
at each energy level is given by Boltzmann's distribution 

Aexp(-s/k B T). 
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Exercise 5 



First, calculate the following: 

at energy 0, exp(—s/k B T) = exp(O) = 1. 
at energy k B T, exp(-s/k B T) = exp(-l) = 0.3679. 
at energy 2k B T, exp(-s/k B T) = exp(-2) = 0.1353. 
at energy 3k B T, exp(-s/k B T) = exp(-3) = 0.0498. 

The sum is 1.553. The fractions are, therefore: 

at energy 0, 1/1.553 = 0.6439. 
at energy k B T, 0.3679/1.553 = 0.2369. 
at energy 2k B T, 0.1353/1.553 = 0.0871. 
at energy 3k B T, 0.0498/1.553 = 0.0321. 

Notice that fraction drops to nearly 1/10 after just 2k B T. 

In statistical mechanics, we often want to know if an energy 
level is likely to be populated at some temperature. If the level 
is much higher than k B T, then it is unlikely to be occupied. 
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